

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>CephFS &amp; RGW Exports over NFS &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="Progress Module" href="../progress/" />
    <link rel="prev" title="MDS Autoscaler Module" href="../mds_autoscaler/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">Ceph 管理器守护进程</a> &raquo;</li>
        
      <li>CephFS &amp; RGW Exports over NFS</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/mgr/nfs.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 管理器守护进程</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../administrator/">安装和配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../modules/">模块编程</a></li>
<li class="toctree-l2"><a class="reference internal" href="../orchestrator_modules/">编写 orchestrator 插件</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dashboard/">仪表盘模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ceph_api/">Ceph RESTful API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../alerts/">Alerts 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../diskprediction/">DiskPrediction 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../localpool/">localpool 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../restful/">RESTful 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../zabbix/">Zabbix 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../prometheus/">Prometheus 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../influx/">Influx 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../hello/">Hello 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../telegraf/">Telegraf 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../telemetry/">Telemetry 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../iostat/">Iostat 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../crash/">Crash 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../insights/">Insights 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../orchestrator/">Orchestrator 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rook/">Rook 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rgw/">RGW 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mds_autoscaler/">MDS Autoscaler 模块</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">NFS 模块</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#nfs-cluster-management">NFS Cluster management</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#create-nfs-ganesha-cluster">Create NFS Ganesha Cluster</a></li>
<li class="toctree-l4"><a class="reference internal" href="#ingress">Ingress</a></li>
<li class="toctree-l4"><a class="reference internal" href="#show-nfs-cluster-ip-s">Show NFS Cluster IP(s)</a></li>
<li class="toctree-l4"><a class="reference internal" href="#delete-nfs-ganesha-cluster">Delete NFS Ganesha Cluster</a></li>
<li class="toctree-l4"><a class="reference internal" href="#updating-an-nfs-cluster">Updating an NFS Cluster</a></li>
<li class="toctree-l4"><a class="reference internal" href="#list-nfs-ganesha-clusters">List NFS Ganesha Clusters</a></li>
<li class="toctree-l4"><a class="reference internal" href="#set-customized-nfs-ganesha-configuration">Set Customized NFS Ganesha Configuration</a></li>
<li class="toctree-l4"><a class="reference internal" href="#view-customized-nfs-ganesha-configuration">View Customized NFS Ganesha Configuration</a></li>
<li class="toctree-l4"><a class="reference internal" href="#reset-nfs-ganesha-configuration">Reset NFS Ganesha Configuration</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#export-management">Export Management</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#create-cephfs-export">Create CephFS Export</a></li>
<li class="toctree-l4"><a class="reference internal" href="#create-rgw-export">Create RGW Export</a></li>
<li class="toctree-l4"><a class="reference internal" href="#delete-export">Delete Export</a></li>
<li class="toctree-l4"><a class="reference internal" href="#list-exports">List Exports</a></li>
<li class="toctree-l4"><a class="reference internal" href="#get-export">Get Export</a></li>
<li class="toctree-l4"><a class="reference internal" href="#create-or-update-export-via-json-specification">Create or update export via JSON specification</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#mounting">Mounting</a></li>
<li class="toctree-l3"><a class="reference internal" href="#troubleshooting">Troubleshooting</a></li>
<li class="toctree-l3"><a class="reference internal" href="#manual-ganesha-deployment">Manual Ganesha deployment</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#known-issues">Known issues</a></li>
<li class="toctree-l4"><a class="reference internal" href="#requirements">Requirements</a></li>
<li class="toctree-l4"><a class="reference internal" href="#ganesha-configuration-hierarchy">Ganesha Configuration Hierarchy</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../progress/">Progress 模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cli_api/">CLI API 命令模块</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="cephfs-rgw-exports-over-nfs">
<span id="mgr-nfs"></span><h1>CephFS &amp; RGW Exports over NFS<a class="headerlink" href="#cephfs-rgw-exports-over-nfs" title="Permalink to this headline">¶</a></h1>
<p>CephFS namespaces and RGW buckets can be exported over NFS protocol
using the <a class="reference external" href="https://github.com/nfs-ganesha/nfs-ganesha/wiki">NFS-Ganesha NFS server</a>.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">nfs</span></code> manager module provides a general interface for managing
NFS exports of either CephFS directories or RGW buckets.  Exports can
be managed either via the CLI <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">nfs</span> <span class="pre">export</span> <span class="pre">...</span></code> commands
or via the dashboard.</p>
<p>The deployment of the nfs-ganesha daemons can also be managed
automatically if either the <a class="reference internal" href="../../cephadm/#cephadm"><span class="std std-ref">Cephadm</span></a> or <a class="reference internal" href="../rook/#mgr-rook"><span class="std std-ref">Rook</span></a>
orchestrators are enabled.  If neither are in use (e.g., Ceph is
deployed via an external orchestrator like Ansible or Puppet), the
nfs-ganesha daemons must be manually deployed; for more information,
see <a class="reference internal" href="#nfs-ganesha-config"><span class="std std-ref">Manual Ganesha deployment</span></a>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Starting with Ceph Pacific, the <code class="docutils literal notranslate"><span class="pre">nfs</span></code> mgr module must be enabled.</p>
</div>
<div class="section" id="nfs-cluster-management">
<h2>NFS Cluster management<a class="headerlink" href="#nfs-cluster-management" title="Permalink to this headline">¶</a></h2>
<div class="section" id="create-nfs-ganesha-cluster">
<h3>Create NFS Ganesha Cluster<a class="headerlink" href="#create-nfs-ganesha-cluster" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster create &lt;cluster_id&gt; <span class="o">[</span>&lt;placement&gt;<span class="o">]</span> <span class="o">[</span>--port &lt;port&gt;<span class="o">]</span> <span class="o">[</span>--ingress --virtual-ip &lt;ip&gt;<span class="o">]</span>
</pre></div>
</div>
<p>This creates a common recovery pool for all NFS Ganesha daemons, new user based on
<code class="docutils literal notranslate"><span class="pre">cluster_id</span></code>, and a common NFS Ganesha config RADOS object.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Since this command also brings up NFS Ganesha daemons using a ceph-mgr
orchestrator module (see <a class="reference internal" href="../orchestrator/"><span class="doc">Orchestrator CLI</span></a>) such as cephadm or rook, at
least one such module must be enabled for it to work.</p>
<p>Currently, NFS Ganesha daemon deployed by cephadm listens on the standard
port. So only one daemon will be deployed on a host.</p>
</div>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is an arbitrary string by which this NFS Ganesha cluster will be
known (e.g., <code class="docutils literal notranslate"><span class="pre">mynfs</span></code>).</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;placement&gt;</span></code> is an optional string signifying which hosts should have NFS Ganesha
daemon containers running on them and, optionally, the total number of NFS
Ganesha daemons on the cluster (should you want to have more than one NFS Ganesha
daemon running per node). For example, the following placement string means
“deploy NFS Ganesha daemons on nodes host1 and host2 (one daemon per host):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="s2">&quot;host1,host2&quot;</span>
</pre></div>
</div>
<p>and this placement specification says to deploy single NFS Ganesha daemon each
on nodes host1 and host2 (for a total of two NFS Ganesha daemons in the
cluster):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="s2">&quot;2 host1,host2&quot;</span>
</pre></div>
</div>
<p>NFS can be deployed on a port other than 2049 (the default) with <code class="docutils literal notranslate"><span class="pre">--port</span> <span class="pre">&lt;port&gt;</span></code>.</p>
<p>To deploy NFS with a high-availability front-end (virtual IP and load balancer), add the
<code class="docutils literal notranslate"><span class="pre">--ingress</span></code> flag and specify a virtual IP address. This will deploy a combination
of keepalived and haproxy to provide an high-availability NFS frontend for the NFS
service.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The ingress implementation is not yet complete.  Enabling
ingress will deploy multiple ganesha instances and balance
load across them, but a host failure will not immediately
cause cephadm to deploy a replacement daemon before the NFS
grace period expires.  This high-availability functionality
is expected to be completed by the Quincy release (March
2022).</p>
</div>
<p>For more details, refer <a class="reference internal" href="../../cephadm/services/#orchestrator-cli-placement-spec"><span class="std std-ref">Daemon Placement</span></a> but keep
in mind that specifying the placement via a YAML file is not supported.</p>
</div>
<div class="section" id="ingress">
<h3>Ingress<a class="headerlink" href="#ingress" title="Permalink to this headline">¶</a></h3>
<p>The core <em>nfs</em> service will deploy one or more nfs-ganesha daemons,
each of which will provide a working NFS endpoint.  The IP for each
NFS endpoint will depend on which host the nfs-ganesha daemons are
deployed.  By default, daemons are placed semi-randomly, but users can
also explicitly control where daemons are placed; see
<a class="reference internal" href="../../cephadm/services/#orchestrator-cli-placement-spec"><span class="std std-ref">Daemon Placement</span></a>.</p>
<p>When a cluster is created with <code class="docutils literal notranslate"><span class="pre">--ingress</span></code>, an <em>ingress</em> service is
additionally deployed to provide load balancing and high-availability
for the NFS servers.  A virtual IP is used to provide a known, stable
NFS endpoint that all clients can use to mount.  Ceph will take care
of the details of NFS redirecting traffic on the virtual IP to the
appropriate backend NFS servers, and redeploying NFS servers when they
fail.</p>
<p>Enabling ingress via the <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">nfs</span> <span class="pre">cluster</span> <span class="pre">create</span></code> command deploys a
simple ingress configuration with the most common configuration
options.  Ingress can also be added to an existing NFS service (e.g.,
one created without the <code class="docutils literal notranslate"><span class="pre">--ingress</span></code> flag), and the basic NFS service can
also be modified after the fact to include non-default options, by modifying
the services directly.  For more information, see <a class="reference internal" href="../../cephadm/services/nfs/#cephadm-ha-nfs"><span class="std std-ref">High-availability NFS</span></a>.</p>
</div>
<div class="section" id="show-nfs-cluster-ip-s">
<h3>Show NFS Cluster IP(s)<a class="headerlink" href="#show-nfs-cluster-ip-s" title="Permalink to this headline">¶</a></h3>
<p>To examine an NFS cluster’s IP endpoints, including the IPs for the individual NFS
daemons, and the virtual IP (if any) for the ingress service,</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster info <span class="o">[</span>&lt;cluster_id&gt;<span class="o">]</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>This will not work with the rook backend. Instead, expose the port with
the kubectl patch command and fetch the port details with kubectl get services
command:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ kubectl patch service -n rook-ceph -p &#39;{&quot;spec&quot;:{&quot;type&quot;: &quot;NodePort&quot;}}&#39; rook-ceph-nfs-&lt;cluster-name&gt;-&lt;node-id&gt;
$ kubectl get services -n rook-ceph rook-ceph-nfs-&lt;cluster-name&gt;-&lt;node-id&gt;
</pre></div>
</div>
</div>
</div>
<div class="section" id="delete-nfs-ganesha-cluster">
<h3>Delete NFS Ganesha Cluster<a class="headerlink" href="#delete-nfs-ganesha-cluster" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster rm &lt;cluster_id&gt;
</pre></div>
</div>
<p>This deletes the deployed cluster.</p>
</div>
<div class="section" id="updating-an-nfs-cluster">
<h3>Updating an NFS Cluster<a class="headerlink" href="#updating-an-nfs-cluster" title="Permalink to this headline">¶</a></h3>
<p>In order to modify cluster parameters (like the port or placement), you need to
use the orchestrator interface to update the NFS service spec.  The safest way to do
that is to export the current spec, modify it, and then re-apply it.  For example,
to modify the <code class="docutils literal notranslate"><span class="pre">nfs.foo</span></code> service,</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph orch ls --service-name nfs.foo --export &gt; nfs.foo.yaml
$ vi nfs.foo.yaml
$ ceph orch apply -i nfs.foo.yaml
</pre></div>
</div>
<p>For more information about the NFS service spec, see <a class="reference internal" href="../../cephadm/services/nfs/#deploy-cephadm-nfs-ganesha"><span class="std std-ref">NFS Service</span></a>.</p>
</div>
<div class="section" id="list-nfs-ganesha-clusters">
<h3>List NFS Ganesha Clusters<a class="headerlink" href="#list-nfs-ganesha-clusters" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster ls
</pre></div>
</div>
<p>This lists deployed clusters.</p>
</div>
<div class="section" id="set-customized-nfs-ganesha-configuration">
<span id="nfs-cluster-set"></span><h3>Set Customized NFS Ganesha Configuration<a class="headerlink" href="#set-customized-nfs-ganesha-configuration" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster config <span class="nb">set</span> &lt;cluster_id&gt; -i &lt;config_file&gt;
</pre></div>
</div>
<p>With this the nfs cluster will use the specified config and it will have
precedence over default config blocks.</p>
<p>Example use cases include:</p>
<ol class="arabic">
<li><p>Changing log level.  The logging level can be adjusted with the following config
fragment:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">LOG</span> <span class="p">{</span>
    <span class="n">COMPONENTS</span> <span class="p">{</span>
        <span class="n">ALL</span> <span class="o">=</span> <span class="n">FULL_DEBUG</span><span class="p">;</span>
    <span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</li>
<li><p>Adding custom export block.</p>
<p>The following sample block creates a single export. This export will not be
managed by <cite>ceph nfs export</cite> interface:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">EXPORT</span> <span class="p">{</span>
  <span class="n">Export_Id</span> <span class="o">=</span> <span class="mi">100</span><span class="p">;</span>
  <span class="n">Transports</span> <span class="o">=</span> <span class="n">TCP</span><span class="p">;</span>
  <span class="n">Path</span> <span class="o">=</span> <span class="o">/</span><span class="p">;</span>
  <span class="n">Pseudo</span> <span class="o">=</span> <span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="p">;</span>
  <span class="n">Protocols</span> <span class="o">=</span> <span class="mi">4</span><span class="p">;</span>
  <span class="n">Access_Type</span> <span class="o">=</span> <span class="n">RW</span><span class="p">;</span>
  <span class="n">Attr_Expiration_Time</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
  <span class="n">Squash</span> <span class="o">=</span> <span class="kc">None</span><span class="p">;</span>
  <span class="n">FSAL</span> <span class="p">{</span>
    <span class="n">Name</span> <span class="o">=</span> <span class="n">CEPH</span><span class="p">;</span>
    <span class="n">Filesystem</span> <span class="o">=</span> <span class="s2">&quot;filesystem name&quot;</span><span class="p">;</span>
    <span class="n">User_Id</span> <span class="o">=</span> <span class="s2">&quot;user id&quot;</span><span class="p">;</span>
    <span class="n">Secret_Access_Key</span> <span class="o">=</span> <span class="s2">&quot;secret key&quot;</span><span class="p">;</span>
  <span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</li>
</ol>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>User specified in FSAL block should have proper caps for NFS-Ganesha
daemons to access ceph cluster. User can be created in following way using
<cite>auth get-or-create</cite>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># ceph auth get-or-create client.&lt;user_id&gt; mon &#39;allow r&#39; osd &#39;allow rw pool=.nfs namespace=&lt;nfs_cluster_name&gt;, allow rw tag cephfs data=&lt;fs_name&gt;&#39; mds &#39;allow rw path=&lt;export_path&gt;&#39;</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="view-customized-nfs-ganesha-configuration">
<h3>View Customized NFS Ganesha Configuration<a class="headerlink" href="#view-customized-nfs-ganesha-configuration" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster config get &lt;cluster_id&gt;
</pre></div>
</div>
<p>This will output the user defined configuration (if any).</p>
</div>
<div class="section" id="reset-nfs-ganesha-configuration">
<h3>Reset NFS Ganesha Configuration<a class="headerlink" href="#reset-nfs-ganesha-configuration" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs cluster config reset &lt;cluster_id&gt;
</pre></div>
</div>
<p>This removes the user defined configuration.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>With a rook deployment, ganesha pods must be explicitly restarted
for the new config blocks to be effective.</p>
</div>
</div>
</div>
<div class="section" id="export-management">
<h2>Export Management<a class="headerlink" href="#export-management" title="Permalink to this headline">¶</a></h2>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Currently, the nfs interface is not integrated with dashboard. Both
dashboard and nfs interface have different export requirements and
create exports differently. Management of dashboard created exports is not
supported.</p>
</div>
<div class="section" id="create-cephfs-export">
<h3>Create CephFS Export<a class="headerlink" href="#create-cephfs-export" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs <span class="nb">export</span> create cephfs --cluster-id &lt;cluster_id&gt; --pseudo-path &lt;pseudo_path&gt; --fsname &lt;fsname&gt; <span class="o">[</span>--readonly<span class="o">]</span> <span class="o">[</span>--path<span class="o">=</span>/path/in/cephfs<span class="o">]</span> <span class="o">[</span>--client_addr &lt;value&gt;...<span class="o">]</span> <span class="o">[</span>--squash &lt;value&gt;<span class="o">]</span>
</pre></div>
</div>
<p>This creates export RADOS objects containing the export block, where</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is the NFS Ganesha cluster ID.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;pseudo_path&gt;</span></code> is the export position within the NFS v4 Pseudo Filesystem where the export will be available on the server. It must be an absolute path and unique.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;fsname&gt;</span></code> is the name of the FS volume used by the NFS Ganesha cluster
that will serve this export.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;path&gt;</span></code> is the path within cephfs. Valid path should be given and default
path is ‘/’. It need not be unique. Subvolume path can be fetched using:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph fs subvolume getpath &lt;vol_name&gt; &lt;subvol_name&gt; [--group_name &lt;subvol_group_name&gt;]
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">&lt;client_addr&gt;</span></code> is the list of client address for which these export
permissions will be applicable. By default all clients can access the export
according to specified export permissions. See the <a class="reference external" href="https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/config_samples/export.txt">NFS-Ganesha Export Sample</a>
for permissible values.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;squash&gt;</span></code> defines the kind of user id squashing to be performed. The default
value is <cite>no_root_squash</cite>. See the <a class="reference external" href="https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/config_samples/export.txt">NFS-Ganesha Export Sample</a> for
permissible values.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Export creation is supported only for NFS Ganesha clusters deployed using nfs interface.</p>
</div>
</div>
<div class="section" id="create-rgw-export">
<h3>Create RGW Export<a class="headerlink" href="#create-rgw-export" title="Permalink to this headline">¶</a></h3>
<p>There are two kinds of RGW exports:</p>
<ul class="simple">
<li><p>a <em>user</em> export will export all buckets owned by an
RGW user, where the top-level directory of the export is a list of buckets.</p></li>
<li><p>a <em>bucket</em> export will export a single bucket, where the top-level directory contains
the objects in the bucket.</p></li>
</ul>
<div class="section" id="rgw-bucket-export">
<h4>RGW bucket export<a class="headerlink" href="#rgw-bucket-export" title="Permalink to this headline">¶</a></h4>
<p>To export a <em>bucket</em>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export create rgw --cluster-id &lt;cluster_id&gt; --pseudo-path &lt;pseudo_path&gt; --bucket &lt;bucket_name&gt; [--user-id &lt;user-id&gt;] [--readonly] [--client_addr &lt;value&gt;...] [--squash &lt;value&gt;]
</pre></div>
</div>
<p>For example, to export <em>mybucket</em> via NFS cluster <em>mynfs</em> at the pseudo-path <em>/bucketdata</em> to any host in the <code class="docutils literal notranslate"><span class="pre">192.168.10.0/24</span></code> network</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --bucket mybucket --client_addr 192.168.10.0/24
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Export creation is supported only for NFS Ganesha clusters deployed using nfs interface.</p>
</div>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is the NFS Ganesha cluster ID.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;pseudo_path&gt;</span></code> is the export position within the NFS v4 Pseudo Filesystem where the export will be available on the server. It must be an absolute path and unique.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;bucket_name&gt;</span></code> is the name of the bucket that will be exported.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;user_id&gt;</span></code> is optional, and specifies which RGW user will be used for read and write
operations to the bucket.  If it is not specified, the user who owns the bucket will be
used.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Currently, if multi-site RGW is enabled, Ceph can only export RGW buckets in the default realm.</p>
</div>
<p><code class="docutils literal notranslate"><span class="pre">&lt;client_addr&gt;</span></code> is the list of client address for which these export
permissions will be applicable. By default all clients can access the export
according to specified export permissions. See the <a class="reference external" href="https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/config_samples/export.txt">NFS-Ganesha Export Sample</a>
for permissible values.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;squash&gt;</span></code> defines the kind of user id squashing to be performed. The default
value is <cite>no_root_squash</cite>. See the <a class="reference external" href="https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/config_samples/export.txt">NFS-Ganesha Export Sample</a> for
permissible values.</p>
</div>
<div class="section" id="rgw-user-export">
<h4>RGW user export<a class="headerlink" href="#rgw-user-export" title="Permalink to this headline">¶</a></h4>
<p>To export an RGW <em>user</em>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export create rgw --cluster-id &lt;cluster_id&gt; --pseudo-path &lt;pseudo_path&gt; --user-id &lt;user-id&gt; [--readonly] [--client_addr &lt;value&gt;...] [--squash &lt;value&gt;]
</pre></div>
</div>
<p>For example, to export <em>myuser</em> via NFS cluster <em>mynfs</em> at the pseudo-path <em>/myuser</em> to any host in the <code class="docutils literal notranslate"><span class="pre">192.168.10.0/24</span></code> network</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --user-id myuser --client_addr 192.168.10.0/24
</pre></div>
</div>
</div>
</div>
<div class="section" id="delete-export">
<h3>Delete Export<a class="headerlink" href="#delete-export" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs <span class="nb">export</span> rm &lt;cluster_id&gt; &lt;pseudo_path&gt;
</pre></div>
</div>
<p>This deletes an export in an NFS Ganesha cluster, where:</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is the NFS Ganesha cluster ID.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;pseudo_path&gt;</span></code> is the pseudo root path (must be an absolute path).</p>
</div>
<div class="section" id="list-exports">
<h3>List Exports<a class="headerlink" href="#list-exports" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs <span class="nb">export</span> ls &lt;cluster_id&gt; <span class="o">[</span>--detailed<span class="o">]</span>
</pre></div>
</div>
<p>It lists exports for a cluster, where:</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is the NFS Ganesha cluster ID.</p>
<p>With the <code class="docutils literal notranslate"><span class="pre">--detailed</span></code> option enabled it shows entire export block.</p>
</div>
<div class="section" id="get-export">
<h3>Get Export<a class="headerlink" href="#get-export" title="Permalink to this headline">¶</a></h3>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph nfs <span class="nb">export</span> info &lt;cluster_id&gt; &lt;pseudo_path&gt;
</pre></div>
</div>
<p>This displays export block for a cluster based on pseudo root name,
where:</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;cluster_id&gt;</span></code> is the NFS Ganesha cluster ID.</p>
<p><code class="docutils literal notranslate"><span class="pre">&lt;pseudo_path&gt;</span></code> is the pseudo root path (must be an absolute path).</p>
</div>
<div class="section" id="create-or-update-export-via-json-specification">
<h3>Create or update export via JSON specification<a class="headerlink" href="#create-or-update-export-via-json-specification" title="Permalink to this headline">¶</a></h3>
<p>An existing export can be dumped in JSON format with:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt1:before {
  content: "# ";
}
</style><span class="prompt1">ceph nfs <span class="nb">export</span> info *&lt;cluster_id&gt;* *&lt;pseudo_path&gt;*</span>
</pre></div></div><p>An export can be created or modified by importing a JSON description in the
same format:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph nfs <span class="nb">export</span> apply *&lt;cluster_id&gt;* -i &lt;json_file&gt;</span>
</pre></div></div><p>For example,:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export info mynfs /cephfs &gt; update_cephfs_export.json
$ cat update_cephfs_export.json
{
  &quot;export_id&quot;: 1,
  &quot;path&quot;: &quot;/&quot;,
  &quot;cluster_id&quot;: &quot;mynfs&quot;,
  &quot;pseudo&quot;: &quot;/cephfs&quot;,
  &quot;access_type&quot;: &quot;RW&quot;,
  &quot;squash&quot;: &quot;no_root_squash&quot;,
  &quot;security_label&quot;: true,
  &quot;protocols&quot;: [
    4
  ],
  &quot;transports&quot;: [
    &quot;TCP&quot;
  ],
  &quot;fsal&quot;: {
    &quot;name&quot;: &quot;CEPH&quot;,
    &quot;user_id&quot;: &quot;nfs.mynfs.1&quot;,
    &quot;fs_name&quot;: &quot;a&quot;,
    &quot;sec_label_xattr&quot;: &quot;&quot;
  },
  &quot;clients&quot;: []
}
</pre></div>
</div>
<p>The imported JSON can be a single dict describing a single export, or a JSON list
containing multiple export dicts.</p>
<p>The exported JSON can be modified and then reapplied.  Below, <em>pseudo</em>
and <em>access_type</em> are modified.  When modifying an export, the
provided JSON should fully describe the new state of the export (just
as when creating a new export), with the exception of the
authentication credentials, which will be carried over from the
previous state of the export where possible.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export apply mynfs -i update_cephfs_export.json
$ cat update_cephfs_export.json
{
  &quot;export_id&quot;: 1,
  &quot;path&quot;: &quot;/&quot;,
  &quot;cluster_id&quot;: &quot;mynfs&quot;,
  &quot;pseudo&quot;: &quot;/cephfs_testing&quot;,
  &quot;access_type&quot;: &quot;RO&quot;,
  &quot;squash&quot;: &quot;no_root_squash&quot;,
  &quot;security_label&quot;: true,
  &quot;protocols&quot;: [
    4
  ],
  &quot;transports&quot;: [
    &quot;TCP&quot;
  ],
  &quot;fsal&quot;: {
    &quot;name&quot;: &quot;CEPH&quot;,
    &quot;user_id&quot;: &quot;nfs.mynfs.1&quot;,
    &quot;fs_name&quot;: &quot;a&quot;,
    &quot;sec_label_xattr&quot;: &quot;&quot;
  },
  &quot;clients&quot;: []
}
</pre></div>
</div>
<p>An export can also be created or updated by injecting a Ganesha NFS EXPORT config
fragment.  For example,:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ ceph nfs export apply mynfs -i update_cephfs_export.conf
$ cat update_cephfs_export.conf
EXPORT {
    FSAL {
        name = &quot;CEPH&quot;;
        filesystem = &quot;a&quot;;
    }
    export_id = 1;
    path = &quot;/&quot;;
    pseudo = &quot;/a&quot;;
    access_type = &quot;RW&quot;;
    squash = &quot;none&quot;;
    attr_expiration_time = 0;
    security_label = true;
    protocols = 4;
    transports = &quot;TCP&quot;;
}
</pre></div>
</div>
</div>
</div>
<div class="section" id="mounting">
<h2>Mounting<a class="headerlink" href="#mounting" title="Permalink to this headline">¶</a></h2>
<p>After the exports are successfully created and NFS Ganesha daemons are
deployed, exports can be mounted with:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ mount -t nfs &lt;ganesha-host-name&gt;:&lt;pseudo_path&gt; &lt;mount-point&gt;
</pre></div>
</div>
<p>For example, if the NFS cluster was created with <code class="docutils literal notranslate"><span class="pre">--ingress</span> <span class="pre">--virtual-ip</span> <span class="pre">192.168.10.10</span></code>
and the export’s pseudo-path was <code class="docutils literal notranslate"><span class="pre">/foo</span></code>, the export can be mounted at <code class="docutils literal notranslate"><span class="pre">/mnt</span></code> with:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ mount -t nfs <span class="m">192</span>.168.10.10:/foo /mnt
</pre></div>
</div>
<p>If the NFS service is running on a non-standard port number:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ mount -t nfs -o <span class="nv">port</span><span class="o">=</span>&lt;ganesha-port&gt; &lt;ganesha-host-name&gt;:&lt;ganesha-pseudo_path&gt; &lt;mount-point&gt;
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Only NFS v4.0+ is supported.</p>
</div>
</div>
<div class="section" id="troubleshooting">
<h2>Troubleshooting<a class="headerlink" href="#troubleshooting" title="Permalink to this headline">¶</a></h2>
<p>Checking NFS-Ganesha logs with</p>
<ol class="arabic">
<li><p><code class="docutils literal notranslate"><span class="pre">cephadm</span></code>: The NFS daemons can be listed with:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ ceph orch ps --daemon-type nfs
</pre></div>
</div>
<p>You can via the logs for a specific daemon (e.g., <code class="docutils literal notranslate"><span class="pre">nfs.mynfs.0.0.myhost.xkfzal</span></code>) on
the relevant host with:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># cephadm logs --fsid &lt;fsid&gt; --name nfs.mynfs.0.0.myhost.xkfzal</span>
</pre></div>
</div>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">rook</span></code>:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ kubectl logs -n rook-ceph rook-ceph-nfs-&lt;cluster_id&gt;-&lt;node_id&gt; nfs-ganesha
</pre></div>
</div>
</li>
</ol>
<p>The NFS log level can be adjusted using <cite>nfs cluster config set</cite> command (see <a class="reference internal" href="#nfs-cluster-set"><span class="std std-ref">Set Customized NFS Ganesha Configuration</span></a>).</p>
</div>
<div class="section" id="manual-ganesha-deployment">
<span id="nfs-ganesha-config"></span><h2>Manual Ganesha deployment<a class="headerlink" href="#manual-ganesha-deployment" title="Permalink to this headline">¶</a></h2>
<p>It may be possible to deploy and manage the NFS ganesha daemons manually
instead of allowing cephadm or rook to do so.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Manual configuration is not tested or fully documented; your
mileage may vary. If you make this work, please help us by
updating this documentation.</p>
</div>
<div class="section" id="known-issues">
<h3>Known issues<a class="headerlink" href="#known-issues" title="Permalink to this headline">¶</a></h3>
<ul class="simple">
<li><p>The <code class="docutils literal notranslate"><span class="pre">mgr/nfs</span></code> module enumerates NFS clusters via the orchestrator API; if NFS is
not managed by the orchestrator (e.g., cephadm or rook) then this will not work.  It
may be possible to create the cluster, mark the cephadm service as ‘unmanaged’, but this
is awkward and not ideal.</p></li>
</ul>
</div>
<div class="section" id="requirements">
<h3>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h3>
<p>The following packages are required to enable CephFS and RGW exports with nfs-ganesha:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">nfs-ganesha</span></code>, <code class="docutils literal notranslate"><span class="pre">nfs-ganesha-ceph</span></code>, <code class="docutils literal notranslate"><span class="pre">nfs-ganesha-rados-grace</span></code> and
<code class="docutils literal notranslate"><span class="pre">nfs-ganesha-rados-urls</span></code> packages (version 3.3 and above)</p></li>
</ul>
</div>
<div class="section" id="ganesha-configuration-hierarchy">
<h3>Ganesha Configuration Hierarchy<a class="headerlink" href="#ganesha-configuration-hierarchy" title="Permalink to this headline">¶</a></h3>
<p>Cephadm and rook start each nfs-ganesha daemon with a minimal
<cite>bootstrap</cite> configuration file that pulls from a shared <cite>common</cite>
configuration stored in the <code class="docutils literal notranslate"><span class="pre">.nfs</span></code> RADOS pool and watches the common
config for changes.  Each export is written to a separate RADOS object
that is referenced by URL from the common config.</p>
</div>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../progress/" class="btn btn-neutral float-right" title="Progress Module" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../mds_autoscaler/" class="btn btn-neutral float-left" title="MDS Autoscaler Module" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>